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ABSTRACT 

A general methodology approach is suggested tor 
measurement of workforce readiness competencies in the context of 
overall work by the National Center for Research on Evaluation, 
Standards, and Student Testing on the domain-independent measurement 
of workforce readiness skills. The methodology consists of lA steps, 
from the initial selection of a work environment to the preparation 
of a report documenting the process. After selection of the work 
environment, a job and task analysis is conducted, and a competency 
or skill is selected that is present in the work environment. 
Component analysis divides the competency into constituent 
subcompetencies for which indicators are created. Indicators are 
classified within a cognitive science taxonomy; and measures of the 
competency are selected or developed, followed by empirical studies, 
statistical analysis, norm creation, and production of a report on 
the indicators. A report on the assessment using multiple indicators 
finally results. Application of the general method to a specific case 
for the Secretary of Labor's Commission on Achieving Necessary Skills 
(SCANS) is documented. Two measures of workforce readiness 
competencies are proposed for the SCANS example of a new restaurant. 
Further development of these measures and empirical validation will 
be conducted in Year 2 of the effort. Twelve tables summarize the 
methodology. An appendix provides the restaurant sce.iario for the 
assessments. (SLD) 
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MEASUREMENT OF WORKFORCE READINESS COMPETENCIES: 
DESIGN OF PROTOTYPE MEASURES 



Hai'old F. O'Neill Jr., CRESST/University of Southern California 
Keith AUred, CRESST/University of California, Los Angeles 
Eva L. Baker, CRESST/University of California, Los Angeles 

Introduction 

The Cognitive Science Laboratory of USC has a subcontract with the 
Center for Research on Evaluation, Standards, and Student Testing (CRESST) 
at the University of California, Los Angeles to assist in the domain- 
independent measurement of workforce readiness skills. In turn, CRESST/ 
UCLA has an existing grant from the Office of Educational Research and 
Improvement to study methodologies for the assessment of competencies 
needed for the workforce. CRESST/UCLA areas of interest include both 
assessment and policy issues. The purpose of this report (Deliverable 3 on our 
USC subcontract) is to provide a context for our work, that is, the Secretary's 
Commission on Achieving Necessary Skills (SCANS), as well as to suggest a 
general methodology approach for measurement of workforce readiness 
competencies that has been instantiated in two measures. 

Workforce Readiness Assessment Methodology 

The methodology that we will document in this report consists of 14 steps, 
from the initial selection of a work environment to the report documenting the 
process (see Table 1). As seen in Table 1, following selection of a work 
environment, a job and task analysis is conducted to determine the 
requirements for the job. Then a competency or skill is selected that is 
assumed or documented to be present in the work environment. Possible 
competencies would be academic (e.g., reading) or interpersonal skills (e.g., 
participating as a member of a team), etc. Unfortunately, such molar 
categories do not map directly onto an assessment measure, and some further 
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Table 1 

Workforce Readiness Assessment Methodology 



• Select a work environment 

• Job and task analysis 

• Select competency 

• Conduct component analysis of competency 

• Create indicator(s) for subcompetencies 

• Classify indicator(s) within a cognitive science taxonomy 

• Create rapid prototypes of measures of indicator(s) test via specifications 

• Select/develop final measures of indicator(s) 

• Select experimental/analytical design 

• Run empirical studies 

• Analyze statistically 

• Use/create norms 

• Report reliability/validity of indicator(s) m'jasure 

• Report on workforce readiness competency using multiple indicators 



level of decomposition is required. Thus, the cognitive analysis step is 
implemented and a component analysis is conducted in order to analyze the 
competency into its constituent subcompetencies. Next, indicators are created 
for the subcompetencies. 

The indicators are then classified within a cognitive science taxonomy. 
The purpose of this step is to allow generalization of the findings from an 
indicator to a high-order subcompetency. Then, measures of the competency 
are selected or developed in two steps: (a) rapid prototypes are developed and 
tested, and (h) prototypes are refined into final measures. Both process and 
outcomes are measured. Next, an experimental/anal3rtical design is selected 
and empirical studies run. The data are statistically analyzed with a focus on 
psychometric issues (e.g., internal consistency, construct validity), and norms 
are used or created. A report on the reliability and validity of the indicator is 
written. Finally, a report on the assessment of the workforce competency 
using multiple indicators is written. 



2 

b 



The application of this general method to a specific case for SCANS will be 
documented in this report and follows in the next section. This report uses 
several analytical approaches as "proof of concept"* of our methodology but does 
not provide an empirical study. Empirical validation will be conducted in 
Year 2 of this effort. 

SCANS 

SCANS is a commission charged by the U.S. Secretary of Labor to 
investigate what is required in today s and tomorrow's workplace and to 
determine the extent to which high school students are able to meet those 
requirements. SCANS was chosen as a target system for our methodology for 
two reasons. First, the SCANS approach includes almost all the competencies 
we are interested in for our assessment approach (for example, SCANS was 
meant to be a national rather than a state or regional assessment). Second, we 
have a good, cooperative relationship with the SCANS staff. 

Specifically, SCANS was directed by the Secretary of Labor to (a) define the 
skills needed for employment, (b) propose acceptable levels of proficiency, 
(c) suggest effective ways to assess proficiency, and (d) develop a dissemination 
strategy for the nation's schools, businesses, and homes. In June 1991, the 
Commission issued a report concerning the first two directives (SCANS, 1991). 
The Commission, based on its discussions and meetings with business 
owners, public employers, unions, and workers and supervisors in shops, 
plants, and stores, identified five competencies in accordance with the first 
directive: the ability to efficiently use (a) resources, (b) interpersonal skills, 
(c) information, (d) systems, and (e) technology (see Table 2). Additionally, the 
Commission found that these five competencies are based on a three-part 
foundation: (a) basic skills, (b) thinking skills, and (c) personal qualities (see 
Table 3). 



Table 2. SCANS: Five Competencies 



Resoiirces: Identines, organizes, plans, and allocates resources 

A. Time — Selects goal-relevant activities, ranks them, allocates time, and 
prepares and follows schedules 

B . Money — Uses or prepares budgets, makes forecasts, keeps records, and 
makes adjustments to meet objectives 

C. Material and Facilities — ^Acquires, stores, allocates, and uses materials or 
space efficiently 

D. Human Resources — ^Assesses skills and distributes work accordingly, 
evaluates performance and provides feedback 

Interpersonal: Works with others 

A. Participates as Member of a Team — Contributes to group effort 

B. Teaches Others New Skills 

C. Serves Clients / Customers — ^Works to satisfy customers' expectations 

D. Exercises Leadership — Communicates ideas to justify position, persuades 
and convinces others, responsibly challenges existing procedures and 
policies 

E. Negotiates — Works toward agreements involving exchange of resources, 
resolves divergent interests 

F. Works with Diversity — ^Works well with men and women from diverse 
backgrounds 

Information: Acquires and uses information 

A. Acquires and Evaluates Information 

B. Organizes and Maintains Information 

C. Interprets and Communicates Information 

D . Uses Computers to Process Information 
Systems: Understands complex inter-relationships 

A. Understands Systems — Knows how social, organizational, and 
technological systems work and operates effectively in them 

B. Monitors and Corrects Performance — Distinguishes trends, predicts 
impacts on system operations, diagnoses deviations in systems' 
performance arid corrects malfunctions 

C. Improves or Designs Systems — Suggests modifications to existing systems 
and develops new or alternative systems to improve performance 

Technology: Works with a variety of technologies 

A. Selects Technology — Chooses procedures, tools or equipment, including 
computers and related technologies 

B. Applies Technology to Task — Understands overall intent and proper 
procedures for setup and operation of equipment 

C. Maintains and Troubleshoots Equipment — Prevents, identifies, or solves 
problems with equipment, including computers and other technologies 



(SCANS, 1991, p. 12) 
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Table 3 

SCANS: A Three-Part Foundation 



Basic Skills: Reads, writes, performs arithmetic and mathematical operations, 
listens and speaks 

A. Reading — Locates, understands, and interprets written information in 
prose and in documents such as manuals, graphs, and schedules 

B. Writing — Communicates thoughts, ideas, information, and messages in 
writing; and creates documents such as letters, directions, manuals, 
reports, graphs, and flow charts 

C. Arithmetic / Mathematics — Performs basic computations and approaches 
practical problems by choosing appropriately from a variety of 
mathematical techniques 

D. Listening — Receives, attends to, interprets, and responds to verbal 
messages and other cues 

E. Speaking — Organizes ideas and communicates orally 

Thinking Skills: Thinks creatively, makes decisions, solves problems, visualizes, 
knows how to learn, and reasons 

A. Creative Thinking — Generates new ideas 

B. Decision Making — Specifies goals and constraints, generates alternatives, 
considers risks, and evaluates and chooses best alternative 

C. Problem Solving — Recognizes problems and devises and implements plan 
of action 

D. Seeing Things in the Mind's Eye — Organizes and processes symbols, 
pictures, graphs, objects, and other information 

E. Knowing How to Learn — Uses efficient learning techniques to acquire and 
apply new knowledge and skills 

F. Reasoning — Discovers a rule or principle underlying the relationship 
between two or more objects and applies it when solving a problem 

Personal Qualities: Displays responsibility, self-esteem, sociability, self-management, 
an(^ integrity and honesty 

A. Responsibility — Exerts a high level of effort and perseveres towards goal 
attainment 

B. Self-Esteem — Believes in own self-worth and maintains a positive view of 
self 

C. Sociability — Demonstrates understanding, friendliness, adaptability, 
empathy, and politeness in group settings 

D. Self "Management — Assesses self accurately, sets personal goals, monitors 
progress, and exhibits self-control 

E. Integrity I Honesty — Chooses ethical courses of action 



(SCANS, 1991, p. 16) 



Although their work with regard to the second directive is not yet 
complete, the Commission also suggested a five-step progression in skills 
acquisition to define proficiency levels for each of the competencies identified. 
The five steps in level of skill acquisition are (a) preparatory, (b) work-ready, 
(c) intermediate, (d) advanced, and (e) specialist. The second level of skills 
acquisition, work-ready y defines the level of proficiency necessary for entry into 
the workforce. Although the Commission has yet to specify the precise levels 
of skill acquisition which would be considered work-ready, it did provide 
examples of how those proficiency levels might look (SCANS, 1991) (see 
Table 4). 

CRESST and its subcontractor, USC, have been asked to participate in the 
third directive, namely to suggest effective ways of assessing proficiency levels 
of the competencies identified by the Commission in its June 1991 report. Our 
report documents progress in the development of an assessment model or 
framework and its instantiation in the assessment of two of the five workforce 
competencies. Specifically, approaches to the assessment of the information 
and interpersonal competencies are suggested. It should be noted that while 
the assessment approaches focus on the two competencies specified, other 
workforce-readiness components identified by the Commission are also 
involved. As the Commission explained in its report, "seldom does one of these 
eight components stand alone in job performance. They are highly integrated 
and most tasks require workers to draw on several of them simultaneously" 
(SCANS, 1991, p. vi). Accordingly, our approach recomjuends the assessment 
of competencies in the context of the foundation skills as well. 

Developing Rapid Prototypes 

The specific approach to developing rapid prototypes for indicators of the 
two competencies of primary focus (information and interpersonal) will be 
described in the following manner. First, the specifications for the sources of 
suggested test content will be elaborated. According to Millman and Greene 
(1989), when tests are designed to assess future performance in a specified 
setting, an analysis of the cognitive requirements of that setting includes two 
steps: 
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Table 4 

Know-How; Work-Ready Level of Proficiency 



COMPETENCE EXAMPIJE OF LEVEL 

RESOURCES Develop cost estimates and write proposals to justify the expense of 

replacing kitchen equipment. Develop schedule for equipment 
delivery to avoid closing restaurant. Read construction blueprints 
and manufacturers' installation requirements to place and install 
equipment in the kitchen.* 

INTERPERSONAL Participate in team training and problem-solving session with 

multi-cultural staff of waiters and waitresses. Focus on upcoming 
Saturday night when local club has reserved restaurant after 
midnight for a party. Three people cannot work and team has to 
address the staffing problem and prepare for handling possible 
complaints about prices, food quality, or service.* 

Analyze statistical control charts to monitor error rate. Develop, 
with other team members, a way to bring performance in production 
line up to that of best practice in competing plants.** 

As part of information analysis above, analyze painting system 
and suggest how improvements can be made to minimize system 
downtime and improve paint finish.** 

Evaluate three new paint spray guns from the point of view of costs, 
health and safety, and speed. Vendors describe performance with 
charts and written specifications. Call vendors' representatives to 
clarify claims and seek the names of others using their equipment. 
Call and interview references before preparing a report on the spray 
guns and make a presentation to management.** 



Progress in Acquiring Skills 



PROFICIENCY LEVEL 


PERI'ORMANCE BENCHMARK 


PREPARATORY 


Scheduling oneself 


WORK-READY 


Scheduling small work team 


INTERMEDIATE 


Scheduling a production line or substantial construction project 


ADVANCED 


Developing roll-out schedule for new product or production plant 


SPECIALIST 


Develop algorithm for scheduling airline 



*Competence as demonstrated in a service sector application, 
**Competence as demonstrated in a manufacturing sector application. 



(Adapted from SCANS, 1991, pp. 26, 28) 



INFORMATION 

SYSTEMS 

TECHNOLOGY 
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Firsts the specific cognitive requirements of the criterion setting are identified^ 
through a job analysis for employment settings. . . . Second^ the content 
specification of the predictive test is developed . . . fcommonly using the] cognitive 
indicators known or hypothesized to be positively related to the criterion 
requirements, (p. 341) 

The Commission, in its study of the workforce, completed the first step in 
Millman and Greene's (1989) process by identifying the cognitive requirements 
for the criterion setting, namely, specific competencies in the workforce, 
SCANS deinved this information via expert judgment, not job analysis. 

In completing the second assessment step, cognitive indicators known or 
hypothesized to be positively related to the criterion requirements are 
identified. The research literature is analyzed to identify these cognitive 
indicators. 

Subsequent to the specification of sources of test-item content, sample test 
items are developed. Finally, specifications for the generation of further test 
items, such as our sample items, are elaborated. The form for the test item 
writing specifications is taken from Baker, Aschbacher, Niemi, Yamaguchi, 
& Ni (1991) and Millman and Greene (1989). 

Because the Commission is currently working to define the work-ready 
proficiency level, it is not possible at this point to provide a precise account of 
how scores on these tests would translate into proficiency levels. Once the 
work-ready proficiency level is specified, pilot studies can be conducted to 
relate the test items to the work-ready proficiency level. 

The Information Competency 

Based upon its discussions and meetings with business owners, public 
employers, unions, and workers and supervisors in shops, plants, and stores, 
the Commission found that the ability to productively use information is 
critical to productivity in the workforce. Technological advances have both 
increased dramatically the amount of information generated and made this 
information potentially more accessible. This explosion in the amount of 
information, along with the rapidity of change in today's workplace, has 
contributed to a heightened need for the efficient use of information. 
Accordingly, the Commission elaborated the cognitive requirements for the 
information competency of the workforce as follows (SCANS, 1991, pp. B1-B2). 
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Acquires and Evaluates Information. Identifies need for data, obtains it 
from existing sources or creates it, and evaluates its relevance and 
accuracy. 

Organizes and Maintains Information. Organizes, processes, and 
maintains written or computerized records and other forms of 
information in a systematic fashion. 

Interprets and Communicates Information. Selects and analyzes 
information and communicates the results to others using oral, written, 
graphic, pictorial, or multi-media methods. 

Uses Computers to Process Information. Employs computers to acquire, 
organize, analyze, and communicate information. 

It is also clear that the information usage competency must be combined 
with the ability to solve problems— one of the important skills identified by the 
Commission as part of the thinking skills foundation. That is, information 
often must be productively used to solve problems based on that information. 

Researchers have examined the cognitive indicators of the intelligent use 
of information in problem solving. In particular, we have drawn upon the 
work of Sternberg (1986) and Mayer, Tajika, and Stanley (1991), who have not 
only examined the cognitive indicators of the intelligent use of information in 
problem solving theoretically and empirically but also developed tests of the 
indicators they have identified. 

Mayer et al. (1991) identified the integration process as one cognitive 
indicator of competence in the intelligent use of information to solve problems. 
The integration process is the ability to identify relevant information, 
distinguishing it from irrelevant information, and then integrate the relevant 
information to solve problems. Table 5 provides a suggested test item for this 
competency. 
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Table 5 

Test Item for Integration Process 



Lucia had $3.00 for lunch. She bought a sandwich 
for $.95, an apple for $.20, and a milk for $.45. How 
much muney did she spend? 



a. 


3.00, 


.95, 


.20. 


.45 


b. 


.95, 


.20, 


.45 




c. 


.95, 


.45 






d. 


3 









(Answer: b) 



(Adapted from Mayer, Tajika, & Stanley, 1991) 



Sternberg (1986) has identified three cognitive processes, similar to Mayer 
et al.'s (1991) integration process, that indicate competence in the intelligent 
use of information in problem solving. First, one must selectively encode the 
information available to solve the problem. In other words, one must identify, 
from the host of information available, which information is relevant to the 
problem at hand and attend to it. Second, one must selectively combine the 
relevant pieces of information. Sternberg observes that although there are 
usually several ways of combining information, there is usually a single 
optimal way for generating a solution to a particular problem. Third, one 
must selectively compare the new information with relevant, previously- 
acquired information in solving problems intelligently. Table 6 provides a 
sample test item for the selective combination aspect of the information and 
thinking skills cognitive indicators suggested by Sternberg (1986). 
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Table 6 

Test Item for Selective Combination 



David is a cook in a small restaurant named "Lester's" which specializes in 
steaks. The restaurant has recently become so popular that the average wait 
to be seated is one hour. Mr. Lester has therefore asked David to reduce the 
amount of time needed to cook an order of six steaks to under one hour. The 
restaurant has a small grill just big enough to broil four steaks at a time. 
David says to himself, "It takes 30 minutes to broil both sides of one steak 
because each side takes 15 minutes. Since I can cook four steaks at the same 
time, 30 minutes will be enough to get four steaks ready. It will take another 
30 minutes to cook the remaining two steaks which means a total of one 
hour." How can David complete cooking all six steaks in just 45 minutes? 

Answer: 

If one combines the information that there are six steaks that take 15 minutes 
per side to broil, in other words, 12 sides to be broiled for 15 minutes, with the 
information that four steaks can be broiled at the same time> one can see that 
if fcur steaks are always on the grill, it will take only 45 minutes to grill all 
six. Assuming the six steaks are grouped into three pairs labeled A, B, and 
C, David can accomplish keeping four steaks on the grill by first broiling 
one side of the two A and two B steaki^, taking off the B steaks and broiling 
side 2 of the A steaks and side 1 of the C steaks, and then broiling side 2 of the 
B and C steaks. 



(Adapted from Sternberg, 1986) 



In summary, as seen in Table 7, we have used Mayer et al. and 
Sternberg's approaches to assessing the information competency, focusing 
primarily (P) on the '^interpreting and communicating data" aspect of the 
information competency combined with both the problem solving skills 
identified as part of the thinking skills foundation competency and the 
arithmetic and mathematics skills identified as part of the basic skills 
foundation competency. A secondary (S) focus in this assessment approach, 
as also seen in Table 7, is the **acquiring and evaluating information" aspect of 
the information competency as well as the creative-thinking and decision- 
making skills identified as parts of the thinking skills foundation. 
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Table 7 

Information Competency 





Acquiring & 
Evaluating 
Information 


Organizing & 
Maintaining 
Information 


Interpreting & 
Communicating 
Information 


Using 
Computers 


Thinking 
Creatively 


S 




s 




Making 
Decisions 


S 




s 




oo 1 ving 
Problems 


S 




P 




Seeing Things 
in Mind's Eye 










Knowing How 
to Learn 










Reasoning 











Reading 










Writing 










Arithmetic 


S 




p 




Mathematics 


s 




p 




Listening 










Speaking 











Note, S = secondary focus, P = primary focus. 



In the next section are item-writing specifications, adapted from Millman 
and Greene (1989, p. 352), for generating sample items such as those above. 



ERIC 



12 

Jl) 



Item-Writing Specifications 

SCANS Competencies to be Tested 
Competencie)s of primary focus. 

1. The information competency: The interpreting and communicating 
information subcompetency, which is the abiUty to select and analyze 
information and to communicate the results to others using oral, 
written, graphic, pictorial, or multi-media methods (SCANS, 1991, 
p. Bl). 

2. The basic skills foundation: 

Arithmetic: Performs basic computations; uses basic numerical 
concepts, such as whole numbers and percentages, in practical 
situations; makes reasonable estimates of arithmetic results without 
a calculator; and uses tables, graphs, diagrams, and charts to obtain 
or convey quantitative information (SCANS, 1991, p. CD. 

Mathematics: Approaches practical problems by choosing 
apjiropriately from a variety of mathematical techniques; uses 
quantitative data to construct logical explanations for real world 
situations; expresses mathematical ideas and concepts orally and in 
writing; and understands the role of chance in the occurrence and 
prediction of events (SCANS, 1991, p. Cl). 

3. The thinking-skills foundation: The problem-solving subskill, which 
is the ability to recognize that a problem exists (i.e., there is a 
discrepancy between what is and what should or could be), to identify 
possible reasons for the discrepancy, and to devise and implement a 
plan of action to resolve it. It also includes the ability to evaluate and 
monitor progress, and to revise the plan as indicated by findings 
(SCANS, 1991, p. C1-C2). 

Competencies of secondary focus: 

1. The information competency: The acquire-and-evaluate-ix)formation 
subcompetency is the ability to identify the need for information, to 
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obtain it from existing sources or to create it, and to evaluate its 
relevance and accuracy (SCANS, 1991, p. Bl). 

2. The thinking skills foundation: 

Creative thinking: Uses imagination freely, combines ideas or 
information in new ways, makes connections between seemingly 
unrelated ideas, and reshapes goals in ways that reveal new 
possibilities (SCANS, 1991, p. CI). 

Decision making: Specifies goals and constraints, generates 
alternatives, considers risks, and evaluates and chooses best 
alternatives (SCANS, 1991, p. Cl). 

Rationale 

Increased volume of information at work and an increased need for 
workers at all levels to be able to use this information effectively require that all 
workers be efficient information managers. 

Content Specification 

Cognitive indicators, either known or hypothesized, of the ability to 
perform the cognitive requirements identified by the Commission should be 
found in relevant research as a source for test content (Millman & Greene, 
1989). In this instance, test content was drawn from the work of Sternberg 
(1986) and Mayer et al. (1991). The specific cognitive indicator drawn from 
Mayer et al. was: 

Integration: The ability to select and combine information into a 
coherent representation of the entire problem. 

The specific cognitive indicator drawn from Sternberg was: 

Selective Combination: The ability to combine information into a 
meaningful whole, in which associations between relevant pieces of 
information are understood, in order to solve a problem. 

Context Attributes 

The context should include a work-place setting involving a problem 
v/hich requires the use of information to solve it. The relevant information can 
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be applied in several possible ways but in only one optimal way for the problem 
involved. 

Question Attributes 

Questions of the following two types can be asked: 

1. Multiple-choice questions in which students must choose the 
multiple-choice option that accurately identifies all and only the 
information relevant to the given problem. 

2. Questions in which the students must identify the relevant 
information and then combine it in the optimal way to generate a 
solution in written form to the given problem- 
Response Attributes 

Responses to the first type of question are simply identification of the one 
correct multiple-choice option. Responses to the second type of question must 
clearly describe in written form the optimal solution to the problem 

Relationship of Scores to SCANS Proficiency Levels 

As mentioned, the Commission is currently working towards greater 
specification of proficiency levels. It is therefore not possible, at this time, to 
specify what levels of performance on these test items would constitute ""work- 
ready" levels of proficiency. Once the Commission has completed its work, 
studies can be conducted to relate scores to proficiency levels. 

Summary of Instantiation of Assessment Methodology 

In the prior section we have instantiated our general methodology (see 
Table 1) in the context of the specific SCANS competency ^^information.'' This 
instantiation is summarized in Table 8. An additional step, ^'Specify basic 
skills foundation," is added in the SCANS case. 




15 



1j 



I 

I 



Table 8 

Workforce Readiness Assessment Methodology for SCANS: Example 1 



General Methodology 



Specific Example 



Select a work environment 
Job and task analysis 
Select competency 



Conduct component analysis of 
competency 



Specify basic skills foundation 

Create indicator(s) for 
subcompetencies 

Classify indicator(s) within a 
cognitive science taxonomy 

Create rapid prototype of measures of 
indicator(s) via test specifications 

Select^develop final measures of 
indicate r(s) 

Select experimental/analytical design 

Run empirical studies 

Analyze statistically 

Use/create norms 

Report reliability/validity of 
indicator(s) measure 

Report on workforce readiness using 
multiple indicators 



Analytically derived 

Analytically derived 

Information 
management (SCANS) 

Interpreting and 
Communicating 
Information 

Arithmetic, Mathematics, 
Thinking Skills 

Integrating process; 
Selective combination 

Mayer et ah, 1991; 
Sternberg, 1986 

[see Table's, Table 6] 

To be done 

Criterion groups 
To be done 
To be done 
To be done 
To be done 

To be done 
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The Interpersonal Competency 

One of the five competencies the Commission identified as critical to 
productive performance in the workforce is interpersonal skills. Specifically, 
interpersonal skills were identified to consist of the ability to participate as a 
member of a team, to teach others, to serve clients and customers, to exercise 
leadership, to negotiate, and to work with cultural diversity (SCANS, 199 1, 
p. Bl). The identification of the interpersonal competency as critical results in 
part from the Commission's finding that a trend exists toward organizing 
workers in terms of teams and toward decision making closer to the front line 
(SCANS, 1991, pp. 3-4). 

The Commission's findings that, to be competitive, America needs to 
organize its workforce in terms of teams that take on problem-solving and 
decision-making responsibilities formerly left to managers further up the 
management hierarchy are confirmed by other commissions and task forces 
examining the skills demands of America's workforce (e.g., Employability 
Skills Task Force, 1989; National Center on Education and the Economy, 1990). 
However, as more tasks and responsibilities are shared and fulfilled 
cooperatively by several persons rather than by individuals acting alone, the 
potential for interpersonal friction increases. 

The Commission we)it on to argue: 

Interpersonal competence is the lubricant of the- workplace, minimizing friction 
and the daily wear and tear of work. It also undergirds restructured work 
organizations in factories and provides the "service" in service firms. It is 
required if teams are to solve problems that they jointly face. All of these competent 
workers function effectively in quite complicated interpersonal environments. A 
false step in most of these situations invites resistance from colleagues or clients 
... (SCANS, 1991, p. 13) 

In elaborating the cognitive requirements of the interpersonal 
competency, the Commission defined the following six subcompetencies 
(SCANS, 1991, p. Bl): 

Participates as a Member of a Team. Works cooperatively with others 
and contributes to group with ideas, suggestions, and effort. 

Teaches Others^ Helps others learn. 

Serves Clients/Customers. Works and communicates with clients and 
customers to satisfy their expectations. 
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Exercises Leadership. Communicates thoughts, feelings, and ideas to 
justify a position; encourages, persuades, convinces, or otherwise 
motivates an individual or groups, including responsibly challenging 
existing procedures, policies, or authority. 

Negotiates. Works towards an agreement that may involve exchanging 
specific resources or resolving divergent interests. 

Works with Cultural Diversity. Works well with men and women and 
with a variety of ethnic, social, or educational backgrounds. 

The Michigan Employability Skills Employer Survey also identified 
several skills related to the interpersonal skills identified by the Commission 
that are critical to the workforce (Mehrens, 1989). The Michigan task force 
administered surveys to a wide variety and large number of employers in the 
workforce to gain their perceptions of the skills required by all workers. Each 
employer was asked to rate 86 skills on a 4-point scale, where 1 = critical, 
2 = highly needed, 3 = somewhat needed, and 4 = not needed. "Pay attention to 
the person speaking" received a mean rating of 1.4. "Ask questions to clarify 
understanding" received a mean rating of 1.5. "Cooperate with others" 
received a mean rating of 1,6. All three skills just mentioned were in the top 20 
of the 86 skills identified on the survey. 

Researchers have examined the cognitive indicators of a number of 
interpersonal competencies that the Commission found critical to productivity 
in the workforce. Specifically, the cognitive indicators of the "negotiates" 
subcompetency, which is defined (SCANS, 1991, p. Bl) as the abihty to work 
towards an agreement that may involve exchanging specific resources or 
resolving divergent interests, are examined in the research on integrative 
negotiation skills (e.g., Womack, 1990). 

Integrative negotiation skills can be defined in terms of three subskills. 
The first subskill is the ability to understand and articulate the common goals 
and interdependency of various colleagues with whom one must work closely 
and cooperatively. Second, integrative negotiation skills include the ability to 
communicate the interests, values, knowledge, and priorities underlying one's 
own suggestions, opinions, and decisions at work, as well as the ability to 
understand and appreciate similar communications from other colleagues 
and team members. Third, integrative negotiation skills include the ability to 
work with others in integrating (thus, the name) the valuable information 
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gained through this effective communication to generate creative solutions to 
problems and tasks encountered at work. 

The three integrative negotiation subskills can also clearly be seen as 
cognitive indicators of the "participates as a member of a team" and "exercises 
leadership" interpersonal subcompetencies, as defined by the Commission. 
The "participates as a member of a team" subcompetency is defined as the 
ability to work cooperatively with others and to contribute to a group with ideas, 
suggestions, and effort (SCANS, 1991, p. Bl). Certainly proficiency in 
integrative negotiation skills indicates the ability to cooperatively share ideas 
and suggestions in group efforts. 

The "exercises leadership" subcompetency is defined as the ability to 
communicate thoughts, feelings, and ideas to justify a position, and to 
encourage, persuade, convince, or otherwise motivate an individual or group, 
including responsibly challenging existing procedures, poHcies, or authority 
(SCANS, 1991, p. Bl). The ability to communicate thoughts, feelings, and ideas 
to justify a position is explicitly demonstrated by proficiency in integrative 
negotiation. One of the major advantages of an integrative approach discussed 
in the literature is to facihtate the responsible challenge of existing policies, 
procedures, or authority. 

As seen in Table 9, our analysis of these interpersonal competencies 
indicates that they also typically draw upon several thinking foundation skills. 
In particular, creative-thinking, decision-making, and problem-solving skills 
(SCANS, 1991, pp. C1-C2) are required for and are indicated by proficiency in 
integrative negotiation. In sum, research on integrative negotiation skills 
documents important cognitive indicators of the interpersonal and thinking 
skills competencies identified by the Commission as critical to performance in 
the workforce. An overview of important research estabUshing these cognitive 
indicators follows. 



ERIC 



19 

2,, 



Table 9 

Interpersonal Skills 





Working 
on Teams 


Teaching 
Others 


Serving 
Customers 


Leading 


Negotiating 


Working 

with 
Cultural 
Diversity 


Thinking 
Creatively 


S 






S 


P 




Making 
Decisions 


S 






S 


P 




Solving 
Problems 


S 






S 


P 




Seeing 
Things in 
Mind's Eye 














Knowing 
How to 
Learn 














Reasoning 















Note, S = secondary focus, P = primary focus. 



One of the most thorough treatments of research on integrative 
negotiation skills is offered in Womack's (1990) review of basic and applied 
research on negotiation of conflict resolution in organizations. Similar to 
other researchers (e.g., Brett, Goldberg, & Ury, 1990), Womack concludes from 
her review that an integrative approach is generally more effective in 
achieving solutions that satisfy the interests and concerns of the colleagues 
involved in problems and disputes, and, therefore, such an approach results in 
longer-lasting resolutions that better meet organization and individual needs. 
She also concludes that integrative negotiation skills contribute to enhanced 
relations between fellow workers. 

Womack (1990) also offers an extensive treatment of research on the 
effectiveness of particular integrative negotiation skills. First, when fellow 
workers or team members are discussing problems over which there are 
divergent opinions, the parties concerned should verbally emphasize their 
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interdependence. Each party should also openly and clearly communicate 
information about its own interests and concerns as well as listen carefully to 
that information from the other party. The verbal communication should also 
include exploratory problem solving, expression of arguments in support of the 
other party's position, and a willingness to accept the other's analysis and 
proposals as legitimate and reasonable. 

Brett et al. (1990) also emphasize the role of integrative communicatioii 
skills in dealing with work situations involving divergent opinions. Based on 
their own as well as others' research, Brett et al. (1990) argue the importance 
of the open exchange of information about each party's interests relevant to the 
dispute rather than communication of principles upon which a party thinks a 
conflict should be resolved. Their argument is that abstract principles and 
rights imperfectly represent the parties' real interests and concerns, thereby 
making resolution of the dispute on such grounds virtually impossible. 

Also, negotiations should include consideration of more than one interest 
at a time so that integrative trade-offs, corresponding to the parties' 
differential priorities, can be discovered. For example, although there may be 
issues where the parties to the dispute have clearly competitive interests, one 
issue might be of high priority to one party, while a different issue is of high 
priority to the other party. If both issues are considered together, each party 
can compromise on the issue of lesser priority and joint positive outcomes can 
be increased. Brett et al. (1990) also emphasize that the mutual dependence of 
the parties should be articulated so that a cooperative rather than a competitive 
orientation prevails whenever possible. 

Brett et al. (1990) argue that when members of the workforce possess and 
use these integrative negotiation skills, solutions will be reached that better 
address the needs, concerns, and interests of the parties within the 
organization as well the needs the organization itself. Furthermore, the 
recurrence of disputes will be diminished. Thus, although an integrative, 
interest-based approach may result in more extended negotiations initially, 
those negotiations will lead to agreements which will hold and prevent later 
conflict. 

Tjosvold (1990) has also addressed the issue of integrative communication 
in the workforce. Tjosvold asked members of a social service organization 
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about recent conflicts they had experienced at work. He found that when 
cooperative goals prevailed and integrative negotiation skills were iised, 
communication was characterized by assistance and support for the interests 
and analyses of the other pa^ y, by a problem-solving orientation, by 
brainstorming for creative solutions for the interests of all parties, and by the 
integration of ideas from the various parties in achieving solutions to work 
problems. Ratings of the effectiveness of conflict resolution and the degree of 
trust in the other party were also high when an integrative orientation 
prevailed. Similar findings were reported by Pruitt and Syna (1983), 

Pruitt and Syna (1983) also review several studies which indicate the 
effectiveness of role reversal, in which each party tries to argue the other side's 
position as accurately and effectively as it can. The results from such an 
approach tend to be superior understanding and greater sympathy for the 
other side's position, and a sense that one's own position had been understood 
and appreciated. 

In summary, the research on integrative negotiation skills provides clear 
documentation of cognitive indicators of important aspects of the interpersonal 
competency identified by the Commission. This stream of research identifies 
the cognitive indicators of the negotiation subcompetency, which is the ability 
to negotiate an agreement involving exchanges of specific resources or 
resolving divergent interests. Integrative negotiation skills also serve as 
cognitive indicators of the ^'participates as a member of a team" and "exercises 
leadership" interpersonal subcompetencies as well as the creative, decision- 
making, and problem-solving thinking skills. According to the integrative 
negotiation skills research, three cognitive indicators of these competencies 
are (a) the ability to identify and articulate the common goals and 
interdependency of the parties, (b) the ability to effectively communicate the 
basis for one's own position as well as to understand and appreciate the other 
party's position(s), and (c) the ability to use the information gained to generate 
creative solutions. 

Drawing upon a workforce scenario (see Table 10) described in the SCANS 
report (SCaNS, 1991, pp. 9«10), sample test items for integrative negotiation 
skills were developed (see Table 11). The SCANS report describes three 
friends' endeavor to open their own restaurant to portray the Commission- 
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Table 10 

Workforce Scenario 



Greg, Anthony, and ICathleen have just embarked upon their entrepreneurial 
dream — opening their own restaurant (The Three Chefs) in a growing southern 
town. Each of them independently worked hard to get to this point, spending 10 or 
more years learning the ropes in the restaurant business, pooling their savings, 
and borrowing from friends and family to get the start-up capital they needed. 

Greg has worked in the restaurant business the longest and has been wanting to 
start his own restaurant for several years so he could be his own boss and enjoy the 
benefits of his own labor. Greg has managed several restaurants and enjoys using 
his business skills to make restaurants a successful business endeavor. Greg put 
up 10% more start-up cash than Anthony and Kathleen and also took out a second 
mortgage on his home to satisfy the local bank's demand for security for operating 
credit. He serves as manager and "front-of-the-house" shift supervisor during the 
day. 

Anthony loves to combine his creative talent with the skills he gained at a culinary 
arts school he attended in the Northeast to produce unique and delicious gourmet 
delights, Anthony trains the staff, does the bookkeeping, and prepares the eveniu^ 
meals, which he loves the most. 

Kathleen has always enjoyed the restaurant business for the service it provides of 
offering a pleasant environment in which family and friends can enjoy a meal 
together, Kathleen majored in interior design in college and enjoys using the 
skills she gained to improve the ambiance of a restaurant. Even when not 
working, Kathleen enjoys going out with friends and family to a restaurant. She is 
the lunchtime chef and evening manager. 



(Adapted from SCANS, 1991, pp. 9-10) 



identified competencies in the accommodations and food services sector of the 
economy. Our modified version (Table 10) expands on each persons 
background and responsibilities and indicates hov^ those differences in 
background and responsibilities led to divergent interests with regard to 
suggestions for improving the restaurant (see Appendix 1 for the original 
scenario). In three different questions, students are asked to perform one of 
the three cognitive indicators. Subsequent to the sample items below, the item- 
writing specifications which were used to generate the items are elaborated. 




23 



2i 



Table 11 

Sample Test Items 



Item 1: Articulate Common Goals 

Write a short essay explaining the goal(s) which Greg, Anthony, and Kathleen have in 
common. Also explain the ways in which they need each other to accomplish their goals. 

Item 2: Understand Others' Positions 

The Three Chefs has been in business for one month. It has been moderately busy in that 
time. During the month, each partner has formed ideas and opinions about how to 
improve The Three Chefs. They have come together on Monday morning to discuss 
their various ideas. They have agreed that they will meet weekly at this time for 45 
minutes to discuss the business. They have also agreed that they will take turns 
chairing the meeting, and it was decided that Greg would chair the first meeting. 

Greg calls first on Anthony, who is hardly able to restrain his enthusiasm to express his 
suggestions. Anthony begins by telling Kathleen and Greg about a new commercial 
food processor he learned of through one of his old classmates at the culinary arts school. 
As Anthony explains different features of the processor, he comments on all the 
wonderful dishes he could prepare with it. He believes that the expense of the equipment, 
which is considerable, will be offset by the volume of business they will do by offering 
such wonderful food and by being able to charge more for it. He therefore suggests that 
they purchase the food processor. His discussion of possible new dishes they could offer 
leads him to a discussion of his other suggestion to order a wider variety of higher 
quality ingredients in order to serve the truly exquisite food he thinks they should offer. 

Afler listening to Anthony's suggestions, Greg asks Kathleen to share her suggestions. 
Kathleen says she has received a few comments from their evening customers that The 
Three Chefs' decor is somewhat barren. Not wanting to expend too much capital without 
having a sense of how good their business was going to be, the three had decided to 
initially buy only the essentials for the restaurant's decor. Kathleen suggests that, 
since they have had an encouraging first month, they should invest in decorating The 
Three Chefs and offers her ideas of some of the improvements that could be made. 
Otherwise, she argues, people won't enjoy eating out at The Three Chefs and won't come 
back. 

Pretend that you are Greg, Anthony and Kathleen have expressed their suggestions as 
described above. It is now your turn, but the 45 minutes allocated for the meeting have 
been taken up. Knowing that everyone has important preparations to make for the 
coming day, you suggest that you will write a summary of Anthony's and Kathleen's 
suggestions as well as your own suggestions rather than explaining them now. It is 
agreed that you will give this summary to Anthony and Kathleen to read before next 
Monday when the three of you will continue the discussion. As Greg, write such a 
summary. You should identify Anthony's and Kathleen's suggestions along with the 
reasons they offered to support those suggestions. You should try to represent Kathleen's 
and Anthony's positions as fairly as possible, while also representing Greg's position, 
as if you were Greg. In representing Greg's position, you should keep the following 
concerns in mind. 
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Table 11 (continued) 



First, given that you put up 10% more start-up capital than the others and took out a 
second mortgage on your home to secure operating credit with the bank, you are quite 
concerned that costs at The Three Chefs be held to a minimum and profits maximized. 
Anthony's and Kathleen's suggestions, all of which sound as though they will involve 
considerable cash outlays, therefore concern you. Second, you have a couple of your own 
suggestions which correspond to your concerns. You have noticed that many of the 
lunchtime customers are in a hurry. On several occasions you heard customers 
commenting that they wished the food would be served more promptly so they wouldn't go 
over their lunch break. You also observed that between 12:00 and 1:00 it was often quite 
busy and that customers had sometimes waited for 2C minutes to be seated. Your idea, 
therefore, is to offer a menu with items that are quicker to prepare to better serve the 
customers. Furthermore, such a change would allow The Three Chefs to do a higher 
volume of business and therefore increase profits. 

Write a short essay that summarizes everyone's position. 

Item 3: Generation of Creative Solutions 

With the information given in item 2, devise a compromise which takes into account 
Greg's, Anthony's, and Kathleen's mutual and individual concerns and suggestions. 
Your goal should be to devise a compromise that best incorporates the information 
provided by the three parties and that best and most fairly accommodates each of their 
positions. Write an essay describing that compromise and how it takes into account the 
concerns and issues raised. 



Item- Writing Sp>ecifications 
SCANS Competencies to be Tested (see Table 9) 
Competencies of primary focus: 

1. The "negotiates** subcompetency of the interpersonal competency: 
Works tov^ards an agreement that may involve exchanging specific 
resources or resolving divergent interests (SCANS, 1991, p. Bl). 

Competencies of secondary focus: 

1. The "exercises leadership** subcompetency of the interpersonal 
competency: Communicates thoughts, feelings, and ideas to justify 
position; encourages, persuades, convinces or otherwise motivates 
an individual or group(s), including responsibly challenging existing 
procedures, policies, or authority (SCANS, 1991, p. Bl). 
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2. The "participates as a member of a team** subcompetency of the 
interpersonal competency: Works cooperatively with others and 
contributes to group with ideas, suggestions and effort. 

3. The creative-thinking subskill of the thinking skills foundation: Uses 
imagination freely, combines ideas or information in new ways, 
makes connections between seemingly unrelated ideas, and reshapes 
goals in ways that reveal new possibilities. 

4. The decision-making subskill of the thinking skills foundation: 
Specifies goals and constraints, generates alternatives, considers 
risks, and evaluates and chooses best alternatives. 

5. The problem-solving subskill of the thinking skills foundation: 
Recognizes that a problem exists (i.e., there is a discrepancy between 
what is and what should or could be), identifies possible reasons for 
the discrepancy, and devises and implements a plan of action to 
resolve it. Evaluates and monitors progress, and revises plan as 
indicated by findings. 

Rationale 

Developments in the ways people are organized at work require more 
interpersonal skills to integrate the input of other members of a team in 
making decisions, thinking creatively, and solving problems to increase 
effectiveness. 

Content Specification 

Cognitive indicators, either known or hypothesized, of the ability to 
perform the cognitive requirements identified by the Commission should be 
found in relevant research as a source for test content (Millman & Greene, 
1989). In this instance, test content was derived from the research on 
integrative negotiation skills. The cognitive indicators from the integrative 
research are summarized as: 

1. The ability to understand and articulate the common goals and 
interdependency of various colleagues with whom one must work 
closely and cooperatively at work. 
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2. The ability to communicate the interests, values, knowledge, and 
priorities underlying one*s own suggestions, opinions, and decisions 
at work, as well as the ability to understand and appreciate similar 
communications from other colleagues and team members. 

3. The ability to work with others in integrating the valuable 
information gained through this effective communication to generate 
creative solutions to problems and tasks encountered at work. 

Context Attributes 

The context should include a work-place setting involving a team of three 
to six members. The context should be presented in the form of a scenario 
describing the work place, a specific task or problem facing the team, and a 
description of the goals, knowledge, concerns, priorities, etc. of each team 
member and the corresponding positions which they take in relation to the 
given task or problem. In addition to the common task or problem, there 
should be some diversity and contradiction in the positions taken by the team 
members. 

Question Attributes 

Three types of essay questions, corresponding to the three cognitive 
indicators, should be asked. 

1. Students should be asked to write a short essay identifying the ways 
in which the team members are dependent on each other to 
accomplish individual and team goals. 



2. 



Students should be asked to fairly represent in essay form, using 
their own words, the positions of the various team members. 
Students should include in their essay the reasons behind each 
member s position. 

3. Students should be asked to integi'ate the information and positions 
offered by each member into a solution which is optimally effective for 
the group as a whole as well as for the individual members. The 
solution, along with an explanation of the ways in which the solution 
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solves the problem and satisfies, to the extent possible, the individual 
members' interests, should be presented in essay form. 

Response Attributes 

Because the responses to the above test items are in essay form, 
specifications for response attributes and scoring have been adapted from 
Baker et al.'s (1991) work in content assessment Baker et al. (1991) elaborate 
an approach to content assessment that involves rating essays along several 
dimensions, with scores ranging from 0 to 5 on each dimension. The relevant 
dimensions are determined by analysis of expert responses to the same test 
items. Baker et al. (1991) elaborate methods of determining scoring 
dimensions as well as methods of training raters and scoring. The 
dimensions implied by the integrative negotiation literature are described 
below, along with scoring guidelines. Studies validating these dimensions and 
guidelines will need to be conducted. 

Item 1: Articulate Common Goals 

1. Generallmpression - Content Quality 

How thoroughly, accurately, and persuasively does the student 
identify and explain the parties' common goals and interdependency? 

(0-5 point global rating: 0=no response, 5=highest level of 
thoroughness, accuracy, and persuasiveness) 

2. Identification of Interdependence of the Parties 

This is a measure of the number of the parties' common goals 
accurately identified by the student (e.g., all three want to own and 
operate their own restaurant successfully) combined with the 
number of ways identified in which the parties are dependent on 
each other to achieve common and individual goals. In the sample 
test item, the parties are dependent upon each other (a) financially — 
only the capital from all three is sufficient, (b) in scheduling — they 
need each other to have people to fulfill the various roles, such as 
manager and chef, during the different shifts, and they need each 
other (c) for their particular expertise and interests, such as 
Anthony's skills as a chef, Kathleen's skills in creating an 
ambiance, and Greg's skills as a business manager. 
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Score Point Guidelines: 

0 = no response 

1 = no common goal or point of interdependence identified 

2 = one common goal or point of interdependence identified 

3 = two common goals or points of interdependence identified 

4 = three common goals or points of interdependence identified 

5 = four common goals or points of interdependence identified 

Item 2: Understand Others' Positions 

1. General Impression - Content Quality 

How fairly and persuasively does the student represent the positions 
of the parties? 

(0-5 point global rating: 0=no response, 5=highest level of fairness 
and persuasiveness) 

2. Nxmiber of Positions Identified 

This is a measure of the extent to which the student identifies each 
party's positions accurately. 

A position in the sample test item is a suggestion for improving The 
Three Chefs. 

The four basic positions in The Three Chefs scenario are: 

1. Anthony's suggestion that they purchase the food processor. 

2. Anthony s suggestion that they purchase higher quality ingredients. 

3. Kathleen's suggestion that they improve the decor. 

4. Greg's position that they offer moderately priced lunches which 
can be served quickly. 

Score Point Guidelines: 

0 = no response 

1 = no positions 

2 = one position 

3 = two positions 

4 = three positions 

5 = four positions 

3. Provides Reasons Behind Each Party's Position(s) 

This is a measure of the extent to which the student documents the 
reasons for the position each party takes. 




An example of a reason in the sample test item would be Kathleen*s 
argument that if the decor is not improved, people will not enjoy 
themselves and won't come back. 

Score Point Guidelines: 

0 = no response 

1 = no reasons 

2 = one reason for one position 

3 = one reason each for two positions 

4 = one reason each for three positions 

5 = one reason each for four positions 



Item 3: Generation of Creative Solutions 

1 . General Impression - Content Quality 

How thoroughly and persuasively does the student identify the 
number of positions taken, fairness, and degree of creative 
integration of positions to generate a solution? 

(0-5 point global rating: 0=no response, 5=highest number of 
positions taken and greatest level of fairness and creativity) 

2. Number of Positions Represented 

How many of the positions in the conflict are represented in the 
solution? 

(0-5 point global rating: 0=no response, l=no positions, 2=one 
position, 3=two positions, 4=three positions, 5=four positions) 

3. Fairness 

Of the positions which are represented, how equally are they 
represented? 

(0-5 point global rating: 0=no response, 5=highest level of fairness) 

4. Creative Integration of the Positions 

This is a measure of the extent to which the creative solution is 
integrative as opposed to distributive. Distributive refers to solutions 
which are simply compromises on some middle ground between 
opposing positions. Integrative refers to creative solutions which 
attempt to integrate positions in such a way that more of the positions 
can be served. 

(0-5 point global rating: 0=no response, 5=highest level of creative 
integrative solution) 
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Example: 



Distributive: Greg wants to cut costs and offer food which can be 
served quicker while Anthony want to buy more efficient 
equipment and more expensive ingredients to offer fancier food. 
The fairest solution then is to just stay with the menu they have 
and not change at all. 

Creative Integrative: Greg wants to increase profits by offering 
food which can be served fastgf at lunch so they can serve more 
customers, while Anthony wants to provide fancier food using 
more expensive ingredients for dinner. The solution is to offer the 
quick, cheaper kinds of food for lunch and the fancier food for 
dinner and charge more for it. 

Relationship of Scores to SCANS Proficiency Levels 

Again, because the Commission's work on specification of proficiency 
levels is ongoing, it is not possible to establish what scores on these items 
would constitute a "work-ready*" level of proficiency. However, once the 
Commission's work is completed, scores will be calibrated to the proficiency 
levels. It is anticipated that the test will be calibrated so that an average score 
of 3 or 4 on each of the dimensions would indicate work-ready competence. 

Summary of Second Instantiation of Assessment Methodology 

In this section we have instantiated our methodology (see Table 1) with a 
second example in the area of interpersonal competence. This instantiation is 
shown in Table 12. As with our first example (see Table 8), an additional step, 
"Specify basic skills foundation,"" is added in the SCANS case. 
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Table 12 

Workforce Readiness Assessment Methodology for SCANS: Example 2 



General Methodology 



Specific Example 



Select a work environment 
Job and task analysis 
Select competency 



Conduct component analysis of 
competency 

Specify basic skills foundation 



Create indicator(s) for subcompetency 
requirement 



Classify indicator(s) within a 
cognitive science taxonomy 

Create rapid prototypes of measures of 
indicator(s) via test specifications 

Select/develop final measures of 
indicator(s) 

Select experimental/analytical design 

Run empirical studies 

Analyze statistically 

Use/create norms 

Report reliability/validity of 
indicator(s) measure 

Report on workforce readiness using 
multiple indicators 



Analytically derived 

Analytically derived 

Interpersonal Competency 
(SCANS) 

Negotiates 



Thinking creatively, making 
decisions, solving problems 

Articulating common goals, 
understanding others' 
positions, creating integrative 
solutions 

Integrative negotiation 
(Womack, 1990) 

[see Tables 10 and 11] 



To be done 

Criterion groups 
To be done 
To be done 
To be done 
To be done 

To be done 
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Where Are We Now? 

We have created a good "first cut" of a general methodology for measuring 
workforce readiness competencies. Further, we have instantiated this 
methodology with two prototypic examples. Our plans are to (a) create the 
assessment measure by increasing the number of test items, (b) conduct an 
empirical study to generate reliability and validity indices, and (c) explore the 
use of technology to administer, score, and interpret our workforce readiness 
competency measure. 
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Appendix 1 

Original Accommodations and Food Services Scenario* 



Greg, Anthony, and Kathleen are on the verge of realizing an 
entrepreneurial dream — opening their own restaurant (The Three Chefs) in a 
gi-owing southern town. Independently, they have worked hard to reach this 
point, spending 10 or more years learning the restaurant business, pooling 
their savings, and borrowing from friends and family to raise the start-up 
capital they needed. Greg took out a second mortgage on his home to satisfy 
the local bank's demand for security on a line of credit. 

Greg serves as manager and ^'front-of-the-house" shift supervisor 
daring the day. Kathleen is the lunchtime chef and evening manager. 
Anthony trains the staff, does the bookkeeping, and prepares the evening 
meals. Renovation has been completed on the restaurant and most of the new 
kitchen equipment has been installed. Waiters and waitresses have completed 
their training and have worked two practice shifts to iron out problems. 

Kathleen and Anthony analyzed the "back-of-the-house" work flow 
during the practice shifts and developed a plan for improving the kitchen's 
output. They can improve efficiency in the kitchen by almost 20 percent by 
starting food preparation f^;* hour early and moving one of the work stations to 
the front of the house. After some discussion, the three of them realized that 
although the repositioning makes sense, it will probably cost them between 
$7,000 and $10,000 which they do not have. If their projections are correct, they 
might be able to afford it after they have made about $250,000 in sales, i.e., in 
three to four months, if all goes well. They opt to make minor adjustments to 
the system and refrain from expensive changes until they have seen how the 
first month's sales and expenses look. 

"Here's another way we can control our costs, says Kathleen, ""'ve 
come across a new management information system that can generate 
inventory reports, sales reports, and pricing charts. We can integrate the 
inventory reports and pricing data to project costs and make menu changes. 
I've also been looking at several different accounting software packages. I 
think the software our accountant recommended is the most suitable for our 
needs. There is a large pool of programmers who know that software, making 
it easier for us to obtain a consultant on short notice to tailor it to our 
operation. 



* From SCANS, What work requires of schools, A SCANS report for America 2000, June 1991, 
pp. 9-10. 
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TIjc following report is condensed 
from CSE/CRESST Technical 
Report 344y Measurement of Work- 
force Readiness Competencies: De- 
sign of Prototype Measures. The 
complete report and a relatedstudyy 
Technical Report 343y Measure- 
ment of Workforce Readiness: Re- 
view of Theoretical FrameworkSy 
are available through the CRESST 
offices. See pa^e 20 to order. 

Current economic difBcul- 
ties and the challenge of 
compering in the world market 
have necessitated a rethinking of 
American approaches to the utili- 
zation of people in organizations. 
Management now recognizes a 
need tc have workers take on 
more responsibility at the points 



of production, of sales, and of 
ser\dce rendered, if the United 
States is to compete in rapidly 
changing world markets. In or- 
der to adapt to the need to intro- 
duce new products and services 
quickly with high quality, new 
directions in manp ^^ement empha- 
size participative management, 
flatter organizational structure, 
just-in-time management, and 
team work. This development 
means that much more is expected 
of even entry-level members of 
the American workforce. 

The need of American man- 
agement for workers with greater 
skills and who can take on greater 
responsibility has spawned many 
commissions, task forces and stud- 
ies. All of them have contributed 



to the vast e\idence documenting 
the need for a more highly skilled 
workforce. What remains largely 
undone is the development of 
methods to assess the necessary 
skills that have been identified. 

The purpose of this report is to 
suggest a general methodology 
for measurement of workforce 
readiness competencies utilizing 
previous research from the 
Secretary's Commission on 
Achieving Necessary Skills 
(SCANS). We have focused on 
the interpersonal competency in 
this article; however, readers in- 
terested in the topic should con- 
sider re\iewingthe two full CSE/ 
CRESST technical reports that 
explain our research in detail. 

(continued on pnjje 2) 
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Workforce Readiness x^ssess- 
ment Methodology- 

The mediodolog}' that we will 
document in diis report consists 
of 14 steps, from the inidal selec- 
tion of a work environment to the 
report viocumenting the process 
(sec Table 1 ), .As seen in Table L 
following selection of a work en- 
vironment, a job and task analysis 
is conducted to determine the 
requirements for the job. Then a 
conipetenc}' or skill is selected 
that is assumed or documented to 
be present in the work environ- 
ment. Possible competencies 
would be academic (e.g., read- 
ing) or interpersonal skills (e.g., 
participating as a member of a 
team), etc. Unfortunately, such 
molar categories do n< map di- 
rccdy onto an assessment mea- 
sure, and some tlirther level of 
decomposition is required. Thus, 
tile cognitive analysis step is imple- 
mented and a component analysis 
is conducted in order to analyze 
the competency into its constitu- 
entsubcompetencies. Next, indi- 
cators are created for the 
subcompetencies. 

The indicators arc then classi- 
fied within a cognitive science 
taxonomy. The purpose of this 
step is to allow generalization of 
the findings fi*om an indicator to 
a high-order subcompetency. 
Then, measures of the compe- 



Table 1 

Workforce Readiness Assessment Methodoloc;\' 



Select a work environment 
Job and task analysis 
Select competency 

Conduct component analysis of competency 

Create indicator(s) for subcompetencies 

Classify' indicator(s) within a cognitive science taxonomy 

Create rapid prototypes of measures of indicator(s) \ia test 

specifications 

Select/develop final measures of indicator(s) 
Select experimental/anahtical design 
Run empirical saidies 
Analyze statistically 
Use/create norms 

Report reliabilit\'/validit\' of indicator(s) measure 
Report on workforce readiness competency using multiple 
indicators 



tency arc selected or developed in 
r^vo steps: (a) rapid protonpes 
are developed and tested, and 
(b) prototypes are refined into 
final measures. Both process and 
outcomes are measured. Next, 
an experimental/analytical design 
is selected and empirical studies 
run. The data are statistically 
analyzed with a focus on psycho- 
metric issues (e.g., internal con- 
sistency, construct validity), and 
norms are used or created. A 
report on the reliability and valid- 
ity of the indicator is wTitten. 



Finally, a report on the assess- 
ment of the workforce compe- 
tency using multiple indicators is 
written. 

The application of this general 
method to a specific case for 
SCiANS will be documented in 
this report and follows in the next 
section. This report uses several 
analytical approaches as "proof of 
concept" of our methodology but 
does not provide an empirical 
study. Empirical validation will 
be conducted in the next year of 
this effort. 
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scans 

scans was a commission 
charged by the U.S. Secretan- of 
Labor to investigate what is re- 
quired in today's and tomorrow's 
workplace and to determine the 
extent to which high schov>l stu- 
dents are able to meet those re- 
quirements. Specihcally, SCANS 
was directed by the Secretan- of 
Labor to ( a ) define the skills need- 
ed for employment, (b) propose 
acceptable levels of proficiency, 
(c) suggest effective ways to as- 
sess proficiency, and (d) develop 
a dissemination strategy' for the 
nation's schools, businesses, and 
homes. 

In June 1991, the Commis- 
sion issued a report concerning 
the first two directives (SCANS, 
1991). The Commission, based 
on its discussions and meetings 
with business owners, public em- 
ployers, unions, and workers and 
super\'isors in shops, plants, and 
stores, identified five competen- 
cies in accordance with the first 
directive: the abil'n- to efficiendy 
use (a) resources, (b) interper- 
sonal skills, (c) information, 
(d) systems, and (e) technology 
(see Table 2). 

(continued on pane 4) 



Table 2 SCANS: Five Competencies i SCANS, IWl, p. ii) 

R.e.sourccs: Identifies, organizes, plans, and allocates resources 

A. r////i*— Selects goal-relevant activities, ranks them, allocates time, 
and prepares and follows schedules 

B. Monn^Vscs or prepares budgets, makes forecasts, keeps records, 
and makes adjustments to meet objecti\'es 

C. Material and Facilitir<— Acquirer, stores, allocates, and usc^ ma- 
terials or space efficiently 

D. Human Resources— Assesses skills and distributes work accord- 
inglv, evaluates performance and provides feedback 

Interpersonal: Works with others 

A. Participates as Member of a Tf/i;//— Contributes to group eftbrt 

B. Teaches Others New Skills 

C. Sores Clients/Customers— Wovks, to sadsfi' customers' expecta- 
tions 

D. Exercises Leadership— Commwnxc^ixcs ideas to justif\* position, 
persuades and convinces others, responsibly challenges existing 
procedures and policies 

E. Nejjotiates— Works coward agreements involving exchange of 
resources, resolves divergent interests 

F Works with Diversit)^\Vorks well with men and women fi-om 
diverse backgrounds 

Information: Acquires and uses information 

A. Acquires and Ei^aluates btformation 

B. Organizes and Maintaiits hiformation 

C. biterpy'ts and Communicates Information 

D. Uses Computers to Process hiformation 

Systems: Understands complex inter-relationships 

A. Understands Systems— Knows how social, organizational, and tech- 
nological systems work and operates etfectively in them 

B. Monitors and Corrects Performance— D\snx\^u\s\\ts trends, pre- 
dicts impacts on system operations, diagnoses de\iations in sys- 
tems' performance and corrects malfunctions 

C. Improves or Designs Systcms—S\i%^csis modifications to existing 
systems and develops new or alternative systems to improve 
performance 
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Table 2 couriumdi SCx\NS: Five Competencies 

Tcchnolog}*: Works with a varict\* ot technologies 

A. Selects TecImohrf\^Chooscs procedures, tools or equipment, including computers and related 
technologies 

B. Applies Technology to Tn^^— Understands o\'erall intent and proper procedures for setup and 
operation of equipment 

C. Maintains and Troublcshoots Equipment— Prevents, identifies, or solves problems with equipment, 
including computers and odier technologies 



CRESST and its subcontrac- 
tor, use, have been asked to 
suggest effecdve ways of assessing 
proticiency levels of the compe- 
tencies identified by tJie Commis- 
sion in its June 1991 report. This 
article documents progress in the 
development of an assessment 
framework and its instantiadon in 
the assessment of die interper- 
sonal competency. It should be 
noted that while our assessment 
approach focuses on the interper- 
sonal competency^ other work- 
force-readiness components iden- 
dfied by the Commission are also 
involved. 



De flu inn the Domain 



Interpersonal Competcnc\' 
Interpersonal skills were iden- 
tified by the Commission to con- 
sist of the abilit}' to participate as 
a member of a team, to teach 
others, to serve clients and cus- 




tomers, to exercise leadership, to 
negodate, nd to work with cul- 
tural diversity^ (SCANS, 1991, 
p.Bl). 

In elaboradng the cognidve 
requirements of the interpersonal 
competency, the Commission 
defined the following six 
subcompetencies (SCANS, June, 
p.Bl): 

Participates as a Member of 
a Team. Works cooperadvely 
with others and contributes to 
group with ideas, suggestions, and 
effort. 

Teaches Others. Helps oth- 
ers learn. 

Serves Clients/Customers. 
Works and communicates with 
clients and customers to satisfy 
their expectations. 

Exercises Leadership. Com- 
municates thoughts, feelings, and 
ideas to justify a position; encour- 
ages, persuades, convinces, or oth- 
erwise motivates an individual or 
groups, including responsibly 



challenging existing procedures, 
policies, or authorit}'. 

Negotiates. VVorks towards 
an agreement that may involve 
exchanging specific resources or 
resolnng divergent interests. 

Works with Cultural Diver- 
sit\\ Works well with men and 
women and with a variet\' of eth- 
nic, social, or educational back- 
grounds. 

Integrative negotiation skills 
can be defined in terms of three 
subskills. The first subskill is die 
abilitv* to understand and articu- 
late the common goals and inter- 
dependency of various colleagues 
with whom one must work closely 
and cooperatively. Second, inte- 
grative negotiation skills include 
the abilit}^ to communicate die 
interests, values, knowledge, and 
priorities undedying one's own 
suggestions, opinions, and deci- 
sions at work, as well as the abilitv- 
to understand and appreciate simi- 
lar communications fi-om other 
colleagues and team members. 
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Tabic 3 

Intcroersonal Skills 










Working 
on Teams 


Teaching 
Others 


Sen'ing 
Customers 


Working with 
Leadinc Neirotiatinsi Cuinira] Diversin* 


Thinking 
Creatively 


S 






S P 


Making 
Dccsions 


s 






S P 


Solving 
Problems 


s 








Seeing 
Things in 
Mind's Eye 










Knowing 
Flow to 
Ixarn 










Reasoning 






Xotc. 


S = secondary locus, P = priinaiy focus. 



Third, integrative negotiation 
skills include the ability' to work 
with others in integrating (thus, 
the name) the valuable informa- 
tion gained through this effective 
communication to generate cre- 
ative solutions to problems and 
tasks encountered at work. 

The three integrative negotia- 
tion subskills can also clearly be 
seen as cognitive indicators of the 
^participates as a member of a 
team" and "exercises leadership'' 
interpersonal subcompetencies, as 
defined by the Commission. 

As seen in Table 3, our analysis 
of these interpersonal competen- 
cies indicates that they also typi- 



cally draw upon several thinking 
foundation skills. In particular, 
creative-thinking, decision-mak- 
ing, and problem-solving skills 
(SCANS, 1991, pp. C1-C2) are 
required for and are indicated by 
proficiency in integrative nego- 
tiation. In sum, research on inte- 
grative negotiation skills docu- 
ments important cognitive indi- 
cators of the interpersonal and 
thinking skills competencies iden- 
tified by the Commission as criti- 
cal to performance in the work- 
force. 

Drawing upon a workforce sce- 
nario (see Table 4) described in 



the SCANS report (SCANS, 
1 99 1 , pp. 9 - 1 0 ), sample test items 
for integrative negotiation skills 
were developed (see Table 5). The 
SCANS report describes three 
friends' endeavor to open their 
own restaurant to poitray the 
Commission-identified compe- 
tencies in the accommodations 
and food services sector of the 
economy. Our modified version 
below expands on each person's 
background and responsibilities 
and indicates how those differ- 
ences in background and respon- 
sibilities led to divergent interests 
(text continued on pajje S) 




/ 
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Workforce Scenario 



Greg, ,-\jithon\\ and Kathleen have just embarked upon their entrepreneurial dream — opening their own 
restaurant (The Three Chefs) in vi growing southern town. Each of them independently worked hard to 
get to this point, spending 10 or more years learning the ropes in the restaurant business, pooling their 
savings, and borrowing from friends and family to get the start-up capital they needed. 

Greg has worked in the restaurant business the longest and has been wanting to start his own restaurant 
for several years so he could be his own boss and enjoy the benefits of his own labor. Greg has managed 
several restaurants and enjoys using his business skills lo make restaurants a successful business endeavor. 
Greg put up 10% more start-up cash than .-Vnthom' and Kathleen and also took out a second mortgage 
on his home to satisfy- the local bank's demand for securit}' for operating credit. He scr\'cs as manager and 
"front-of- the -house" shift supen'isor during the day. 

Anthom' loves to combine his creative talent with the skills he gained at a culinarv^ ans school h.: attended 
in the Northeast to produce unique and delicious gourmet delights. Anthony trains the staff, does the 
bookkeeping, and prepares the evening meals, which he loves the most. 

Kathleen has always enjoyed the restaurant business for the senice it provides of offering a pleasant 
cn\'ironment in which family and friends can enjoy a meal together. Kathleen majored in interior design 
in college and enjoys using the skills she gained to improve the ambiance of a restaurant. Even when not 
working, Kathleen enjoys going out with friends and family to a restaurant. She is the lunch time chef and 
evening manager. 



Table 5 

Sample Test Items 
Item 1: Articulate Common Goals 

Write a short essay explaining the goal(s) which Greg, Aiithony, and Kathleen have in common. Also 
explain the ways in which they need each other to accomplish their goals. 

Item 2: Understand Otliers' Positions 

The Three Chefs has been in business for one month. It has been moderately busy in that time. During 
the month, each partner has formed ideas and opinions about how to improve The Three Chefs. They 
have come together on Monday morning to discuss tlieir various ideas. They have agreed that they will 
meet weekly at this time for 45 minutes to discuss the business. They have also agreed that they will take 
turns chairing the meeting, and it was decided that Greg would chair the first meeting. 

Greg calls first on Anthony, who is hardly able to restrain his enthusiasm to express his suggestions. 
Anthony begins by telling Kathleen and Greg about a new commercial food processor he learned of 
through one of his old classmates at the culinary arts school. As Anthony explains different features of the 
processor, he comments on all the wonderful dishes he could prepare with it. He believes that the expense 
of the equipment, which is considerable, will be offset by the volume of business they wall do by offering 
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such wondcrtiil food and by being able to charge more for it. He therefore suggests that they purchase 
the food processor, His discussion of possible new dishes they could ofter leads him to a discussion of his 
other suggestion to order a wider variety of higher quality ingredients in order to sen'c the truly exquisite 
food he thinks they should offer, 

After listening to Anthony's suggestions, Greg asks Kathleen to share her suggestions. Kathleen says she 
has received a few comments from their evening customers that The Three Chefs' decor is somewhat 
barren. Not wanting to expend too much capital withcHit ha\*ing a sense of how good their business was 
going to be, the three had decided to initially buy only the essentials for the restaurant's decor. Kathleen 
suggests that, since they have had an encouraging first month, they should invest in decorating The Three 
Chefs and offers her ideas of some of the improvements that could be made. Othcr\\'ise, she argues, people 
won't enjoy eating out at The Three Chefs and won't come back. 

Pretend that you are Greg. Anthony and Kathleen have expressed their suggestions as described abo\*c. 
It is now your turn, but the 45 minutes allocated for the meeting have been taken up. Knowing that 
e\'er\'one has important preparations to make for the coming day, you suggest that you wiW write a summaiy 
of Anthony's and Kathleen's suggestions as well as your own suggestions rather than explaining them now. 
It is agreed that you will give this summary to Anthony and Kathleen to read before next Monday when 
the three of you will continue the discussion. As Greg, write such a summar\'. You should identity' 
Anthony's and Kathleen's suggestions along with the reasons the; offered to support those suggestions. 
You should tr\' to represent Kathleen's and Anthony's positions as fairly as possible, while also representing 
Greg's position, as if you were Greg. In representing Greg's position, you should keep the following 
concerns in mind. 

First, given that you put up 10% more start-up capital than the others and took out a second mortgage on 
your home to secure operating credit with the bank, you are quite concerned that costs at The Three Chets 
be held to a minimum and profits maximized. Anthony's and Kathleen's suggestions, all of which sound 
as tliough they will involve considerable cash oudays, therefore concern you. Second, you have a couple 
ofyour own suggestions which correspond to your concerns. You have noticedthat many ofthc lunchtime 
customers arc in a hurt)'. On se\'eral occasions you heard customers commenting that they wished the food 
would be scn*cd more prompdy so they wouldn't go over their lunch break. You also obser\xd that 
between 12:00 and 1:00 it was often quite busy and diat customers had sometimes waited for 20 minutes 
to be seated. Your idea, therefore, is to offer a menu with items which are quicker to prepare to better ser\*e 
the customers. Furthermore, such a change would allowThe Three Chefs to do a higher volume of business 
and therefore increase profits. 

Write a short essay that summarizes ever^'one's posidon. 
Item 3: Generation of Creative Solutions 

With the inforniauon given in item 2, de\ise a compromise which takes into account Greg's, Anthony's, 
and Kathleen's mutual and individual concerns and suggestions. Your goal should be to devise a 
compromise which best incorporates the informadon provided by the three parties and which best and most 
fairly accommodates each of their positions. Write an essay describing that compromise and how it takes 
into account the concerns and issues raised. 
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(text from piifie 5) 
with regard to suggestions for 
improving the restaurant. In diree 
difterent questions, students are 
asked to perform one of the three 
cognitive indicators. Following 
are the item-writing specitications 
which were used to generate the 
sample items in Table 5. 



IruM-W Ri riNG Specifications 



SCxVNS Competencies to be 

Tested isec Table 2) 

Competencies of prim aj-y focus: 
The ''negotiates'' subcom- 
petency of the interpersonal 
competenq': Works towards 
an agreement that may in- 
volve exchangingspecific re- 
sources or resolving diver- 
gent interests (SCANS, 
1991, p. Bl). 

Competencies of secondary focus: 
L The ''exercises leadership'' 
subcompetency of die inter- 
personal competenq': Com- 
municates thoughts, feel- 
ings, and ideas to justify po- 
sition; encourages, per- 
suades, convinces or other- 
wise motivates an individual 
or group(s), including re- 
sponsibly challenging exist- 
ing procedures, policies, or 
authority (SCANS, June 
1991, p. Bl). 



2. The ''participates as a mem- 
ber of a team" subcompe- 
tency of the interpersonal 
competency: Works coop- 
eratively with odiers and con- 
tributes to group with ideas, 
suggestions, and etVort. 

3. The crcadve-thinking sub- 
skill of the thinking skills 
foundation: Uses imagina- 
tion freely, combines ideas 
or information in new ways, 
makes connections between 
seemingly unrelated ideas, 
and reshapes goals in ways 
that reveal new possibilities. 

4. The decision-making subskill 
of the thinking skills founda- 
tion: Specifies goals and 
constraints, generates alter- 
natives, considers risks, and 
evaluates and chooses best 
alternatives. 

5. Theproblem-soKingsubskill 
of the thinking skills founda- 
don: Recognizes thataprob- 
lem exists (i.e., there is a 
discrepancy between what is 
and what should or could 
be), identifies possible rea- 
sons for the discrepancy, and 
devises and implements a 
plan of action to resolve it. 
Evaluates and monitors 
progress, and revises plan as 
indicated by findings. 



Rationale 

Developments in the ways 
people are organized at work re- 
quire more interpersonal skills to 
integrate the input of other mem- 
bers of a team in making deci- 
sions, thinkingcreatively, and solv- 
ing problems to increase eflec- 
tiveness. 

Content Specification 

Cognitive indicators, either 
known or hypothesized, of the 
abilit\' to perform the cognitive 
requirements identified by the 
Commission should be found in 
relevant research as a source for 
test content (Millman &: Greene, 
1989). In this instance, test con- 
tent was derived from the research 
on integrative negotiation skills. 
The cognitive indicators from die 
integrative research are summa- 
rized as: 

1 . The abilit}' to understand aiid 
articulate the common goals 
and interdepcndency ofvari- 
ous colleagues with whom 
one must work closely and 
cooperatively at work. 

2. The abilitv' to communicate 
the interests, values, knowl- 
edge, and priorities underly- 
ing one's own suggestions, 
opinions, and decisions at 
work, as well as the ability^ to 
understand and appreciate 
similar communications 
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from other colleagues and 
team members. 
3 . The abilit>' to work w ith oth- 
ers in integrating the valu- 
able information gained 
through this effective com- 
munication to generate cre- 
ative solutions to problems 
and tasks encountered at 
work. 

Context Attributes 

The context should include a 
work-place setting involving a 
team of three to six members. 
The context should be presented 
in the form of a scenario describ- 
ing the work place, a specific task 
or problem facing the team, and a 
description of the goals, knowl- 
edge, concerns, priorities, etc. of 
each team member and the corre- 
sponding posidons which they 
take in relation to the given task 
or problem. In addition to the 
common task or problem, there 
should be some diversit\^and con- 
tradiction in the positions taken 
by the team members. 

Question Attributes 

Three t\'pes of essay questions, 
corresponding to the three cog- 
nitive indicators, should be asked. 

1 . Students should be asked to 
write a short essay identif}'- 
ing the ways in which the 
team members are depen- 



dent on each other to ac- 
complish indi\idual and team 
goals. 

2. Students should be asked to 
fairly represent in essay form, 
using their own words, the 
positions of the \ anous team 
members. Students should 
include in their essay the rea- 
sons behind each member's 
position. 

3. Students should be asked to 
integrate the information 
and positions offered by each 
member into a solution 
which is optimally effective 
for the group as a whole as 
well as for the individual 
members. The solution, 
along with an explanation of 
the ways in which the solu- 
tion solves the problem and 
satisfies, to the extent pos- 
sible, the individual mem- 
bers' interests, should be 
presented in essay form. 

Response Attributes 

Because the responses to the 
above test items are in essay form, 
specifications for response at- 
tributes and scoring have been 
adapted from Baker et al . 's (1 99 1 ) 
workincontentasscssment. Baker 
et al. ( 1 99 1 ) elaborate an approach 
to content assessment that in- 
volves rating essays along several 
dimensions, with scores ranging 
from 0 to 5 on each dimension. 



The relevant dimensions are de- 
termined by analysis of expert re- 
sponses to the same test items. 
Baker et al. (1991) elaborate 
mediods of determining scoring 
dimensions as well as methods of 
training raters and scoring. The 
dimensions implied by the inte- 
grative negotiation literature are 
described below, along with scor- 
ing guidelines. Studies validating 
these dimensions and guidelines 
will need to be conducted. 

Item 1: Articulate Common 
Goals 

1 . General Impression -Con- 
tent Qualit}' 

How dioroughiy, accurately, 
and persuasively does the student 
identif\' and explain the parties' 
common goals and interdepen- 
dency.> (0-5 point global rating: 
0=no response, 5-highest level of 
thoroughness, accuracy, and per- 
suasiveness) 

2. Identification of Interde- 
pendence of the Parties 

This is a measure of the num- 
ber of the parties' common goals 
accurately idendfied by the stu- 
dent (e.g., all three want to own 
and operate their own restaurarit 
succcssftilly) combined with the 
numberofwaysidenufiedin which 
the parties are dependent on each 
other to achieve common and 
individual goals. In the sample 
test item, the parties are depen- 



'<9 



pauc 10 



Measurement of Workforce Readiness Competencies: 
Design of Prototype Measures 



dent upon each other 
(a) financially — only the capital 
from all three is sufficient, (b) in 
scheduling— they need each other 
to have people to tiilHll the vari- 
ous roles, such as manager and 
chet, during the different shifts, 
and they need each other (c) for 
their particular expertise and in- 
terests, such as Anthony's skills as 
a chef, Kathleen's skills in creat- 
ing an ambiance, and Greg's skills 
as a business manager. 

Score Point Guidelines: 
0= no response 

1= no common goal or point of 
interdependence identified 

2= one common goal or point 
of interdependence identi- 
fied 

3= two common goals or points 
of interdependence identi- 
fied 

4= three common goals or 
points of interdependence 
identified 

5= four common goals or points 
of interdependence identi- 
fied 

Item 2: Understand Others' 
Positions 
1 . General Impression ~ Con- 
tent Quality 
How fairly and persuasively 
does the student represent the 
positions of the parties? (0-5 point 
global rating: 0=no response, 



5=highest level of fairness and 
persuasiveness) 
2. Number of Positions Iden- 
tified 

This is a measure of the extent 
to which the student identifies 
each part\''s positions accurately. 
A position in the sample test item 
is a suggestion for improx'ing The 
Three Chefs. The four basic po- 
sitions are: 

1. Anthony's suggestion that 
they purchase the food pro- 
cessor, 

2. Anthony's suggestion diat 
they purchase higher quality 
ingredients. 

3. Kathleen's suggestion that 
they improve the decor. 

4. Greg's position that they 
offer moderately priced 
lunches which can be served 
quickly. 

Score Point Guidelines: 
0= no response 
1= no positions 
2= one position 
3= t\vo positions 
4= three positions 
5= four positions 

3. Provides Reasons Behind 
Each Part}''s Positions 
This is a measure of the extent 
to which the student documents 
the reasons for the position each 
part)' takes. 

An example of a reason in the 



sample test item would be 
Kathleen's argument that if the 
decor is not improved, people will 
not enjoy themselves and won't 
come back. 

Score Point Guidelines: 

0- no rc<;ponsc 
1 '-^ nci reasons 

2= one reason for one position 
3= one reason each for two po- 
sitions 

4= one reason each for three 

positions 
5= one reason each for four 

positions 

Item 3: Generation of 
Creative Solutions 

1 . General Impression - Con- 
tent Qualit}* 

How thoroughly and persua- 
sively does the student identifv' 
the number of positions taken, 
fairness, and degree of creative 
integration of positions to gener- 
ate a solution? (0-5 point global 
rating: 0=no response, 5=highest 
number of positions taken and 
greatest level of fairness and cre- 
ativity) 

2, Number of Positions Rep- 
resented 

How many of the positions in 
the conflict are represented in the 
solution? (0-5 point global rat- 
ing: 0=no response, l=no posi- 
tions, 2=one position, 3=t\vo po- 
sitions, 4=tiiree positions, 5= four 
positions) 
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3. Fairness 

Ot the positions which arc rep- 
resented, how equally are they 
represented? 

1 0-5 point global rating: 0=no 
response, 5=highest level of fair- 
ness I 

4. Creative Integration of the 
Positions 

This is a measure of the extent 
to which the creative solution is 
integrative as opposed to distribu - 
tive. Distributive retcrs to solu- 
tions which are simply compro- 
mises on some middle ground 
between opposing positions. In- 
tegrative retcrs to creative solu- 
tions which attempt to integrate 
positions in such a way that more 
of the positions can be ser\-ed. (0- 
5 point global rating: 0=no re- 
sponse, 5=highest level of cre- 
ative integrative solution) 

Example: 
Distributive: Greg wants to cut 
costs and oftcr food which can be 
ser\'ed quicker while Anthony 
want to buy more efficient equip- 
ment and more expensive ingre- 
dients to ofter fancier food. The 
fairest solution then is to just stay 
with the menu they have and not 
change at all. 

Creative Integrative: Greg wants 
to increase profits by offering food 
which can beser\'ed faster at lunch 
so they can ser\'c more customers, 
while Anthony wants to pro\dde 
fancier food using more expen- 
sive ingredients for dinner. The 



solution is to offer the quick, 
cheaper kinds of food for lunch 
and the fancier food for dinner 
and charge more for it. 

Relationship of Scores to 
scans Proficiency Levels 

Again, because the Commis- 
sion's work on specification of 
proficiency levels was not com- 
plete at the time of our original 
report, it is not possible to estab- 
lish what scores on these items 
would constitute a "work -ready'"* 
level of proficiency. However, 
once the Commission's work is 
completed, scores will be cali- 
brated to the proficiency levels. It 
is anticipated that the test will be 
calibrated so that an average score 
of 3 or 4 on each of the dimen- 
sions would indicate work- ready 
competence. 

Simiman,' of Assessment Meth- 
odology 

In this report we have applied 
amcthodolog\'(sccTable I) with 
an example in the area of interper- 
sonal competence. A work envi- 
ronment (restaurant) and job and 
task analysis were analytically de- 
rived. The interpersonal compe- 
tency was selected, basic skills 
specififed, indicators created and 
a prototype measure developed. 



Where Are Wc Now? 

We have created a good "first 
cut'' of a general methodolog\' 
for measuring workforce readi- 
ness competencies. Further, we 
have instantiated this methodol- 
ogy with two prototypic examples 
(one discussed in this repoit ). Our 
future plans are to (a) create the 
assessment measure by increasing 
the number of test items, 
(b) conduct an empirical study to 
generate reliability* and vahdity 
indices, and (c) explore the use of 
technology to administer, score, 
and interpret our workforce readi - 
ness competency measure, j^- 

References 

Baker, E.L., Aschbacher, P.E., 
Niemi, D., Yamaguchi, E., & 
Ni, Y. (1991). Cognitively 
sensitive assessments of student 
writing in content areas. Los 
Angeles: University of Cali- 
fornia, Center for Research on 
Evaluation, Standards, and Stu- 
dent Testing. 

xMillman, J., & Greene, J. ( 1 989). 
The specification and develop- 
ment of tests of achievement 
andabilit\^ InR.L. Linn(Ed.), 
Educational measurement (3rd 
cd., pp. 335-366). New York: 
Macmillan. 

SCANS. (1991, June). What 
work requires of schools. A 
SCANS report for America 
2000. Washington, DC: U.S. 
Department of Labor, The 
Secretary's Commission on 
Achieving Necessarv Skills 
(SCANS). 




^1 



